Background-Aware Classification Activation Map for Weakly Supervised Object Localization.

Weakly supervised object localization (WSOL) relaxes the requirement of dense annotations for object localization by using image-level annotation to supervise the learning process. However, most WSOL methods only focus on forcing the object classifier to produce high activation score on object parts without considering the influence of background locations, causing excessive background activations and ill-pose background score searching. Based on this point, our work proposes a novel mechanism called the background-aware classification activation map (B-CAM) to add background awareness for WSOL training. Besides aggregating an object image-level feature for supervision, our B-CAM produces an additional background image-level feature to represent the pure-background sample. This additional feature can provide background cues for the object classifier to suppress the background activations on object localization maps. Moreover, our B-CAM also trained a background classifier with image-level annotation to produce adaptive background scores when determining the binary localization mask. Experiments indicate the effectiveness of the proposed B-CAM on four different types of WSOL benchmarks, including CUB-200, ILSVRC, OpenImages, and VOC2012 datasets.

Discriminative ensemble meta-learning with co-regularization for rare fundus diseases diagnosis.

Deep neural networks (DNNs) have been widely applied in the medical image community, contributing to automatic ophthalmic screening systems for some common diseases. However, the incidence of fundus diseases patterns exhibits a typical long-tailed distribution. In clinic, a small number of common fundus diseases have sufficient observed cases for large-scale analysis while most of the fundus diseases are infrequent. For these rare diseases with extremely low-data regimes, it is challenging to train DNNs to realize automatic diagnosis. In this work, we develop an automatic diagnosis system for rare fundus diseases, based on the meta-learning framework. The system incorporates a co-regularization loss and the ensemble-learning strategy into the meta-learning framework, fully leveraging the advantage of multi-scale hierarchical feature embedding. We initially conduct comparative experiments on our newly-constructed lightweight multi-disease fundus images dataset for the few-shot recognition task (namely, FundusData-FS). Moreover, we verify the cross-domain transferability from miniImageNet to FundusData-FS, and further confirm our method's good repeatability. Rigorous experiments demonstrate that our method can detect rare fundus diseases, and is superior to the state-of-the-art methods. These investigations demonstrate that the potential of our method for the real clinical practice is promising.

SGLSA: Sphygmus gated laser speckle angiography for microcirculation hemodynamics imaging.

Hemodynamics imaging of the retinal microcirculation has been demonstrated to be potential access to evaluating ophthalmic diseases, cardio-cerebrovascular diseases, and metabolic diseases. However, existing structural and functional imaging techniques are insufficient in spatial or temporal resolution. The sphygmus gated laser speckle angiography (SGLSA) is proposed for structural and functional imaging with high spatiotemporal resolution. Compared with classic LSCI algorithms, SGLSA presents a much clearer perfusion image and higher signal-to-noise ratio pulsatility. The SGLSA algorithm also shows better performance on patients than traditional LSCI methods. The high spatiotemporal resolution provided by the SGLSA algorithm greatly enhances the ability of retinal microcirculation analysis, which makes up for the deficiency of the LSCI technology, and attaches great significance to retinal hemodynamic imaging, biomarker research, and clinical diagnosis.

Bayesian statistics-guided label refurbishment mechanism: Mitigating label noise in medical image classification.

PURPOSE: Deep neural networks (DNNs) have been widely applied in medical image classification, benefiting from its powerful mapping capability among medical images. However, these existing deep learning-based methods depend on an enormous amount of carefully labeled images. Meanwhile, noise is inevitably introduced in the labeling process, degrading the performance of models. Hence, it is significant to devise robust training strategies to mitigate label noise in the medical image classification tasks. METHODS: In this work, we propose a novel Bayesian statistics-guided label refurbishment mechanism (BLRM) for DNNs to prevent overfitting noisy images. BLRM utilizes maximum a posteriori probability in the Bayesian statistics and the exponentially time-weighted technique to selectively correct the labels of noisy images. The training images are purified gradually with the training epochs when BLRM is activated, further improving classification performance. RESULTS: Comprehensive experiments on both synthetic noisy images (public OCT & Messidor datasets) and real-world noisy images (ANIMAL-10N) demonstrate that BLRM refurbishes the noisy labels selectively, curbing the adverse effects of noisy data. Also, the anti-noise BLRMs integrated with DNNs are effective at different noise ratio and are independent of backbone DNN architectures. In addition, BLRM is superior to state-of-the-art comparative methods of anti-noise. CONCLUSIONS: These investigations indicate that the proposed BLRM is well capable of mitigating label noise in medical image classification tasks.

Triplet Cross-Fusion Learning for Unpaired Image Denoising in Optical Coherence Tomography.

Optical coherence tomography (OCT) is a widely-used modality in clinical imaging, which suffers from the speckle noise inevitably. Deep learning has proven its superior capability in OCT image denoising, while the difficulty of acquiring a large number of well-registered OCT image pairs limits the developments of paired learning methods. To solve this problem, some unpaired learning methods have been proposed, where the denoising networks can be trained with unpaired OCT data. However, majority of them are modified from the cycleGAN framework. These cycleGAN-based methods train at least two generators and two discriminators, while only one generator is needed for the inference. The dual-generator and dual-discriminator structures of cycleGAN-based methods demand a large amount of computing resource, which may be redundant for OCT denoising tasks. In this work, we propose a novel triplet cross-fusion learning (TCFL) strategy for unpaired OCT image denoising. The model complexity of our strategy is much lower than those of the cycleGAN-based methods. During training, the clean components and the noise components from the triplet of three unpaired images are cross-fused, helping the network extract more speckle noise information to improve the denoising accuracy. Furthermore, the TCFL-based network which is trained with triplets can deal with limited training data scenarios. The results demonstrate that the TCFL strategy outperforms state-of-the-art unpaired methods both qualitatively and quantitatively, and even achieves denoising performance comparable with paired methods. Code is available at: https://github.com/gengmufeng/TCFL-OCT.

Synergistically segmenting choroidal layer and vessel using deep learning for choroid structure analysis.

Objective. The choroid is the most vascularized structure in the human eye, whose layer structure and vessel distribution are both critical for the physiology of the retina, and disease pathogenesis of the eye. Although some works have used graph-based methods or convolutional neural networks to separate the choroid layer from the outer-choroid structure, few works focused on further distinguishing the inner-choroid structure, such as the choroid vessel and choroid stroma.Approach.Inspired by the multi-task learning strategy, in this paper, we propose a segmentation pipeline for choroid analysis which can separate the choroid layer from other structures and segment the choroid vessel synergistically. The key component of this pipeline is the proposed choroidal U-shape network (CUNet), which catches both correlation features and specific features between the choroid layer and the choroid vessel. Then pixel-wise classification is generated based on these two types of features to obtain choroid layer segmentation and vessel segmentation. Besides, the training process of CUNet is supervised by a proposed adaptive multi-task segmentation loss which adds a regularization term that is used to balance the performance of the two tasks.Main results.Experiments show the high performance (4% higher dice score) and less computational complexity (18.85 M lower size) of our proposed strategy.Significance.The high performance and generalization on both choroid layer and vessel segmentation indicate the clinical potential of our proposed pipeline.

Rethinking the neighborhood information for deep learning-based optical coherence tomography angiography.

PURPOSE: Optical coherence tomography angiography (OCTA) is a premium imaging modality for noninvasive microvasculature studies. Deep learning networks have achieved promising results in the OCTA reconstruction task, benefiting from their powerful modeling capability. However, two limitations exist in the current deep learning-based OCTA reconstruction methods: (a) the angiogram information extraction is only limited to the locally consecutive B-scans; and (b) all reconstruction models are confined to the 2D convolutional network architectures, lacking effective temporal modeling. As a result, the valuable neighborhood information and inherent temporal characteristics of OCTA are not fully utilized. In this paper, we designed a neighborhood information-fused Pseudo-3D U-Net (NI-P3D-U) for OCTA reconstruction. METHODS: The proposed NI-P3D-U was investigated on an in vivo animal dataset by a cross-validation strategy under both fully supervised learning and weakly supervised learning pipelines. To demonstrate the OCTA reconstruction capability of the proposed NI-P3D-U, we compared it with several state-of-the-art methods. RESULTS: The results showed that the proposed network outperformed the state-of-the-art deep learning-based OCTA algorithms in terms of visual quality and quantitative metrics, and demonstrated an effective generalization for different training strategies (fully supervised and weakly supervised) and imaging protocols. Meanwhile, the idea of neighborhood information fusion was also expanded to other network architectures, resulting in significant improvements. CONCLUSIONS: These investigations indicate that the proposed network, which combines the neighborhood information strategy with temporal modeling architecture, is well capable of performing OCTA reconstruction, and has a certain potential for clinical applications.

Content-Noise Complementary Learning for Medical Image Denoising.

Medical imaging denoising faces great challenges, yet is in great demand. With its distinctive characteristics, medical imaging denoising in the image domain requires innovative deep learning strategies. In this study, we propose a simple yet effective strategy, the content-noise complementary learning (CNCL) strategy, in which two deep learning predictors are used to learn the respective content and noise of the image dataset complementarily. A medical image denoising pipeline based on the CNCL strategy is presented, and is implemented as a generative adversarial network, where various representative networks (including U-Net, DnCNN, and SRDenseNet) are investigated as the predictors. The performance of these implemented models has been validated on medical imaging datasets including CT, MR, and PET. The results show that this strategy outperforms state-of-the-art denoising algorithms in terms of visual quality and quantitative metrics, and the strategy demonstrates a robust generalization capability. These findings validate that this simple yet effective strategy demonstrates promising potential for medical image denoising tasks, which could exert a clinical impact in the future. Code is available at: https://github.com/gengmufeng/CNCL-denoising.

Comparative study of deep neural networks with unsupervised Noise2Noise strategy for noise reduction of optical coherence tomography images.

As a powerful diagnostic tool, optical coherence tomography (OCT) has been widely used in various clinical setting. However, OCT images are susceptible to inherent speckle noise that may contaminate subtle structure information, due to low-coherence interferometric imaging procedure. Many supervised learning-based models have achieved impressive performance in reducing speckle noise of OCT images trained with a large number of noisy-clean paired OCT images, which are not commonly feasible in clinical practice. In this article, we conducted a comparative study to investigate the denoising performance of OCT images over different deep neural networks through an unsupervised Noise2Noise (N2N) strategy, which only trained with noisy OCT samples. Four representative network architectures including U-shaped model, multi-information stream model, straight-information stream model and GAN-based model were investigated on an OCT image dataset acquired from healthy human eyes. The results demonstrated all four unsupervised N2N models offered denoised OCT images with a performance comparable with that of supervised learning models, illustrating the effectiveness of unsupervised N2N models in denoising OCT images. Furthermore, U-shaped models and GAN-based models using UNet network as generator are two preferred and suitable architectures for reducing speckle noise of OCT images and preserving fine structure information of retinal layers under unsupervised N2N circumstances.

PMS-GAN: Parallel Multi-Stream Generative Adversarial Network for Multi-Material Decomposition in Spectral Computed Tomography.

Spectral computed tomography is able to provide quantitative information on the scanned object and enables material decomposition. Traditional projection-based material decomposition methods suffer from the nonlinearity of the imaging system, which limits the decomposition accuracy. Inspired by the generative adversarial network, we proposed a novel parallel multi-stream generative adversarial network (PMS-GAN) to perform projection-based multi-material decomposition in spectral computed tomography. By designing the differential map and incorporating the adversarial network into loss function, the decomposition accuracy was significantly improved with robust performance. The proposed network was quantitatively evaluated by both simulation and experimental study. The results show that PMS-GAN outperformed the reference methods with certain robustness. Compared with Pix2pix-GAN, PMS-GAN increased the structural similarity index by 172% on the contrast agent Ultravist370, 11% on bones, and 71% on bone marrow, respectively, in a simulated test scenario. In an experimental test scenario, 9% and 38% improvements of the structural similarity index on the biopsy needle and on a torso phantom were observed, respectively. The proposed network demonstrates its capability of multi-material decomposition and has certain potential toward clinical applications.

Weakly Supervised Deep Learning-Based Optical Coherence Tomography Angiography.

Optical coherence tomography angiography (OCTA) is a promising imaging modality for microvasculature studies. Deep learning networks have been widely applied in the field of OCTA reconstruction, benefiting from its powerful mapping capability among images. However, these existing deep learning-based methods depend on high-quality labels, which are hard to acquire considering imaging hardware limitations and practical data acquisition conditions. In this article, we proposed an unprecedented weakly supervised deep learning-based pipeline for OCTA reconstruction task, in the absence of high-quality training labels. The proposed pipeline was investigated on an in vivo animal dataset and a human eye dataset by a cross-validation strategy. Compared with supervised learning approaches, the proposed approach demonstrated similar or even better performance in the OCTA reconstruction task. These investigations indicate that the proposed weakly supervised learning strategy is well capable of performing OCTA reconstruction, and has a certain potential towards clinical applications.