Bulk-Switching Memristor-Based Compute-In-Memory Module for Deep Neural Network Training.

The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor-based compute-in-memory (CIM) modules can perform vector-matrix multiplication (VMM) in place and in parallel, and have shown great promises in DNN inference applications. However, CIM-based model training faces challenges due to non-linear weight updates, device variations, and low-precision. In this work, a mixed-precision training scheme is experimentally implemented to mitigate these effects using a bulk-switching memristor-based CIM module. Low-precision CIM modules are used to accelerate the expensive VMM operations, with high-precision weight updates accumulated in digital units. Memristor devices are only changed when the accumulated weight update value exceeds a pre-defined threshold. The proposed scheme is implemented with a system-onchip of fully integrated analog CIM modules and digital sub-systems, showing fast convergence of LeNet training to 97.73%. The efficacy of training larger models is evaluated using realistic hardware parameters and verifies that CIM modules can enable efficient mix-precision DNN training with accuracy comparable to full-precision software-trained models. Additionally, models trained on chip are inherently robust to hardware variations, allowing direct mapping to CIM inference chips without additional re-training.

Performance Evaluation of Deep Learning Models on Mammogram Classification Using Small Dataset.

Cancer is the second leading cause of death globally, and breast cancer (BC) is the second most reported cancer. Although the incidence rate is reducing in developed countries, the reverse is the case in low- and middle-income countries. Early detection has been found to contain cancer growth, prevent metastasis, ease treatment, and reduce mortality by 25%. The digital mammogram is one of the most common, cheapest, and most effective BC screening techniques capable of early detection of up to 90% BC incidence. However, the mammogram is one of the most difficult medical images to analyze. In this paper, we present a method of training a deep learning model for BC diagnosis. We developed a discriminative fine-tuning method which dynamically assigns different learning rates to each layer of the deep CNN. In addition, the model was trained using mixed-precision training to ease the computational demand of training deep learning models. Lastly, we present data augmentation methods for mammograms. The discriminative fine-tuning algorithm enables rapid convergence of the model loss; hence, the models were trained to attain their best performance within 50 epochs. Comparing the results, DenseNet achieved the highest accuracy of 0.998, while AlexNet obtained 0.988.

Optimal hyperparameter selection of deep learning models for COVID-19 chest X-ray classification.

The first and most critical response to curbing the spread of the novel coronavirus disease (COVID-19) is to deploy effective techniques to test potentially infected patients, isolate them and commence immediate treatment. However, several test kits currently in use are slow and in a shortage of supply. This paper presents techniques for diagnosing COVID-19 from chest X-ray (CXR) and address problems associated with training deep models with less voluminous datasets and class imbalance as obtained in most available CXR datasets on COVID-19. We used the discriminative fine-tuning approach, which dynamically assigns different learning rates to each layer of the network. The learning rate is set using the cyclical learning rate policy that changes per iteration. This flexibility ensured rapid convergence and avoided being stuck in saddle point plateau. In addition, we addressed the high computational demand of deep models by implementing our algorithm using the memory- and computational-efficient mixed-precision training. Despite the availability of scanty datasets, our model achieved high performance and generalisation. A Validation accuracy of 96.83%, sensitivity and specificity of 96.26% and 95.54% were obtained, respectively. When tested on an entirely new dataset, the model achieves 97% accuracy without further training. Lastly, we presented a visual interpretation of the model's output to prove that the model can aid radiologists in rapidly screening for the symptoms of COVID-19.

Hybrid Data Fusion DBN for Intelligent Fault Diagnosis of Vehicle Reducers.

Given its importance, fault diagnosis has attracted considerable attention in the literature, and several machine learning methods have been proposed to discover the characteristics of different aspects in fault diagnosis. In this paper, we propose a Hybrid Deep Belief Network (HDBN) learning model that integrates data in different ways for intelligent fault diagnosis in motor drive systems, such as a vehicle drive system. In particular, we propose three data fusion methods: data union, data join, and data hybrid, based on detailed data fusion research. Additionally, the significance of the fusion is explained from the energy perspective of the signal. In particular, the appropriate fusion methods and data structures suitable for model training requirements can help improve the accuracy of fault diagnosis. Moreover, mixed-precision training is used as a special fusion method to further improve the performance of the model. Experiments with the datasets obtained from the simulation platform demonstrate the superiority of our proposed model over the state-of-the-art methods.