RETHINKING INTERMEDIATE LAYERS DESIGN IN KNOWLEDGE DISTILLATION FOR KIDNEY AND LIVER TUMOR SEGMENTATION.

Knowledge distillation (KD) has demonstrated remarkable success across various domains, but its application to medical imaging tasks, such as kidney and liver tumor segmentation, has encountered challenges. Many existing KD methods are not specifically tailored for these tasks. Moreover, prevalent KD methods often lack a careful consideration of 'what' and 'from where' to distill knowledge from the teacher to the student. This oversight may lead to issues like the accumulation of training bias within shallower student layers, potentially compromising the effectiveness of KD. To address these challenges, we propose Hierarchical Layer-selective Feedback Distillation (HLFD). HLFD strategically distills knowledge from a combination of middle layers to earlier layers and transfers final layer knowledge to intermediate layers at both the feature and pixel levels. This design allows the model to learn higher-quality representations from earlier layers, resulting in a robust and compact student model. Extensive quantitative evaluations reveal that HLFD outperforms existing methods by a significant margin. For example, in the kidney segmentation task, HLFD surpasses the student model (without KD) by over 10%, significantly improving its focus on tumor-specific features. From a qualitative standpoint, the student model trained using HLFD excels at suppressing irrelevant information and can focus sharply on tumor-specific details, which opens a new pathway for more efficient and accurate diagnostic tools. Code is available here.

Mott memristor based stochastic neurons for probabilistic computing.

Many studies suggest that probabilistic spiking in biological neural systems is beneficial as it aids learning and provides Bayesian inference-like dynamics. If appropriately utilised, noise and stochasticity in nanoscale devices can benefit neuromorphic systems. In this paper, we build a stochastic leaky integrate and fire (LIF) neuron, utilising a Mott memristor's inherent stochastic switching dynamics. We demonstrate that the developed LIF neuron is capable of biological neural dynamics. We leverage these characteristics of the proposed LIF neuron by integrating it into a population-coded spiking neural network and a spiking restricted Boltzmann machine (sRBM), thereby showcasing its ability to implement probabilistic learning and inference. The sRBM achieves a software-comparable accuracy of 87.13%. Unlike CMOS-based probabilistic neurons, our design does not require any external noise sources. The designed neurons are highly energy efficient and ultra-compact, requiring only three components: a resistor, a capacitor and a memristor device.

CLINet: A novel deep learning network for ECG signal classification.

Machine learning is poised to revolutionize medicine with algorithms that spot cardiac arrhythmia. An automated diagnostic approach can boost the efficacy of diagnosing life-threatening arrhythmia disorders in routine medical procedures. In this paper, we propose a deep learning network CLINet for ECG signal classification. Our network uses convolution, LSTM and involution layers to bring their unique advantages together. For both convolution and involution layers, we use multiple, large size kernels for multi-scale representation learning. CLINet does not require complicated pre-processing and can handle electrocardiograms of any length. Our network achieves 99.90% accuracy on the ICCAD dataset and 99.94% accuracy on the MIT-BIH dataset. With only 297 K parameters, our model can be easily embedded in smart wearable devices. The source code of CLINet is available at https://github.com/CandleLabAI/CLINet-ECG-Classification-2024.

PaCL: Patient-aware contrastive learning through metadata refinement for generalized early disease diagnosis.

Early diagnosis plays a pivotal role in effectively treating numerous diseases, especially in healthcare scenarios where prompt and accurate diagnoses are essential. Contrastive learning (CL) has emerged as a promising approach for medical tasks, offering advantages over traditional supervised learning methods. However, in healthcare, patient metadata contains valuable clinical information that can enhance representations, yet existing CL methods often overlook this data. In this study, we propose an novel approach that leverages both clinical information and imaging data in contrastive learning to enhance model generalization and interpretability. Furthermore, existing contrastive methods may be prone to sampling bias, which can lead to the model capturing spurious relationships and exhibiting unequal performance across protected subgroups frequently encountered in medical settings. To address these limitations, we introduce Patient-aware Contrastive Learning (PaCL), featuring an inter-class separability objective (IeSO) and an intra-class diversity objective (IaDO). IeSO harnesses rich clinical information to refine samples, while IaDO ensures the necessary diversity among samples to prevent class collapse. We demonstrate the effectiveness of PaCL both theoretically through causal refinements and empirically across six real-world medical imaging tasks spanning three imaging modalities: ophthalmology, radiology, and dermatology. Notably, PaCL outperforms previous techniques across all six tasks.

A Survey of Deep Learning Techniques for Underwater Image Classification.

In recent years, there has been an enormous interest in using deep learning to classify underwater images to identify various objects, such as fishes, plankton, coral reefs, seagrass, submarines, and gestures of sea divers. This classification is essential for measuring the water bodies' health and quality and protecting the endangered species. Furthermore, it has applications in oceanography, marine economy and defense, environment protection, underwater exploration, and human-robot collaborative tasks. This article presents a survey of deep learning techniques for performing underwater image classification. We underscore the similarities and differences of several methods. We believe that underwater image classification is one of the killer application that would test the ultimate success of deep learning techniques. Toward realizing that goal, this survey seeks to inform researchers about state-of-the-art on deep learning on underwater images and also motivate them to push its frontiers forward.

FEEDNet: a feature enhanced encoder-decoder LSTM network for nuclei instance segmentation for histopathological diagnosis.

Objective.Automated cell nuclei segmentation is vital for the histopathological diagnosis of cancer. However, nuclei segmentation from 'hematoxylin and eosin' (HE) stained 'whole slide images' (WSIs) remains a challenge due to noise-induced intensity variations and uneven staining. The goal of this paper is to propose a novel deep learning model for accurately segmenting the nuclei in HE-stained WSIs.Approach.We introduce FEEDNet, a novel encoder-decoder network that uses LSTM units and 'feature enhancement blocks' (FE-blocks). Our proposed FE-block avoids the loss of location information incurred by pooling layers by concatenating the downsampled version of the original image to preserve pixel intensities. FEEDNet uses an LSTM unit to capture multi-channel representations compactly. Secondly, for datasets that provide class information, we train a multiclass segmentation model, which generates masks corresponding to each class at the output. Using this information, we generate more accurate binary masks than that generated by conventional binary segmentation models.Main results.We have thoroughly evaluated FEEDNet on CoNSeP, Kumar, and CPM-17 datasets. FEEDNet achieves the best value of PQ (panoptic quality) on CoNSeP and CPM-17 datasets and the second best value of PQ on the Kumar dataset. The 32-bit floating-point version of FEEDNet has a model size of 64.90 MB. With INT8 quantization, the model size reduces to only 16.51 MB, with a negligible loss in predictive performance on Kumar and CPM-17 datasets and a minor loss on the CoNSeP dataset.Significance.Our proposed idea of generalized class-aware binary segmentation is shown to be accurate on a variety of datasets. FEEDNet has a smaller model size than the previous nuclei segmentation networks, which makes it suitable for execution on memory-constrained edge devices. The state-of-the-art predictive performance of FEEDNet makes it the most preferred network. The source code can be obtained fromhttps://github.com/CandleLabAI/FEEDNet.

A Survey of Deep Learning on CPUs: Opportunities and Co-Optimizations.

CPU is a powerful, pervasive, and indispensable platform for running deep learning (DL) workloads in systems ranging from mobile to extreme-end servers. In this article, we present a survey of techniques for optimizing DL applications on CPUs. We include the methods proposed for both inference and training and those offered in the context of mobile, desktop/server, and distributed systems. We identify the areas of strength and weaknesses of CPUs in the field of DL. This article will interest practitioners and researchers in the area of artificial intelligence, computer architecture, mobile systems, and parallel computing.