Hardware implementation of memristor-based artificial neural networks.

Artificial Intelligence (AI) is currently experiencing a bloom driven by deep learning (DL) techniques, which rely on networks of connected simple computing units operating in parallel. The low communication bandwidth between memory and processing units in conventional von Neumann machines does not support the requirements of emerging applications that rely extensively on large sets of data. More recent computing paradigms, such as high parallelization and near-memory computing, help alleviate the data communication bottleneck to some extent, but paradigm- shifting concepts are required. Memristors, a novel beyond-complementary metal-oxide-semiconductor (CMOS) technology, are a promising choice for memory devices due to their unique intrinsic device-level properties, enabling both storing and computing with a small, massively-parallel footprint at low power. Theoretically, this directly translates to a major boost in energy efficiency and computational throughput, but various practical challenges remain. In this work we review the latest efforts for achieving hardware-based memristive artificial neural networks (ANNs), describing with detail the working principia of each block and the different design alternatives with their own advantages and disadvantages, as well as the tools required for accurate estimation of performance metrics. Ultimately, we aim to provide a comprehensive protocol of the materials and methods involved in memristive neural networks to those aiming to start working in this field and the experts looking for a holistic approach.

Changes in morphological traits, anatomical and molecular alterations caused by gamma-rays and zinc oxide nanoparticles in spinach (Spinacia oleracea L.) plant.

Spinach seeds were irradiated with gamma-rays after that soaked in zinc oxide nanoparticles (ZnO-NPs) at 0.0, 50, 100 and 200 ppm for twenty-four hours at room temperature. Vegetative plant growth, photosynthetic pigments, and proline contents were investigated. Also, anatomical studies and the polymorphism by the SCoT technique were conducted. The present results revealed that the germination percentage was at the maximum values for the treatment of 100 ppm ZnO-NPs (92%), followed by 100 ppm ZnO-NPs + 60 Gy (90%). The application of ZnO-NPs resulted in an enhancement in the plant length. The maximum of chlorophylls and carotenoids content was recorded in the treatment, 100 ppm ZnO-NPs + 60 Gy. Meanwhile, the irradiation dose level (60 Gy) with all ZnO-NPs treatments increased proline content and reached its maximum increase to 1.069 mg/g FW for the treatment 60 Gy combined with 200 ppm ZnO-NPs. Also, the anatomical studies declared that there were variations between the treatments; un-irradiated and irradiated combined with ZnO-NPs plants which reveal that the leave epidermal tissue increased with 200 ppm ZnO-NPs in both the upper and lower epidermis. While irradiated plants with 60 Gy combined with 100 ppm ZnO-NPs gave more thickness of upper epidermis. As well as SCoT molecular marker technique effectively induced molecular alterations between the treatments. Where, SCoT primers targeted many new and missing amplicons that are expected to be associated with the lowly and highly expressed genes with 18.2 and 81.8%, respectively. Also, showed that the soaking in ZnO-NPs was helped for reducing molecular alteration rate, both spontaneous and induced by gamma irradiation. This nominates ZnO-NPs as potential nano-protective agents that can reduce irradiation-induced genetic damage.

Efficient training of spiking neural networks with temporally-truncated local backpropagation through time.

Directly training spiking neural networks (SNNs) has remained challenging due to complex neural dynamics and intrinsic non-differentiability in firing functions. The well-known backpropagation through time (BPTT) algorithm proposed to train SNNs suffers from large memory footprint and prohibits backward and update unlocking, making it impossible to exploit the potential of locally-supervised training methods. This work proposes an efficient and direct training algorithm for SNNs that integrates a locally-supervised training method with a temporally-truncated BPTT algorithm. The proposed algorithm explores both temporal and spatial locality in BPTT and contributes to significant reduction in computational cost including GPU memory utilization, main memory access and arithmetic operations. We thoroughly explore the design space concerning temporal truncation length and local training block size and benchmark their impact on classification accuracy of different networks running different types of tasks. The results reveal that temporal truncation has a negative effect on the accuracy of classifying frame-based datasets, but leads to improvement in accuracy on event-based datasets. In spite of resulting information loss, local training is capable of alleviating overfitting. The combined effect of temporal truncation and local training can lead to the slowdown of accuracy drop and even improvement in accuracy. In addition, training deep SNNs' models such as AlexNet classifying CIFAR10-DVS dataset leads to 7.26% increase in accuracy, 89.94% reduction in GPU memory, 10.79% reduction in memory access, and 99.64% reduction in MAC operations compared to the standard end-to-end BPTT. Thus, the proposed method has shown high potential to enable fast and energy-efficient on-chip training for real-time learning at the edge.

Machine Learning-Based Unobtrusive Intake Gesture Detection via Wearable Inertial Sensors.

Dietary patterns can be the primary reason for many chronic diseases such as diabetes and obesity. State-of-the-art wearable sensor technologies can play a critical role in assisting patients in managing their eating habits by providing meaningful statistics on critical parameters such as the onset, duration, and frequency of eating. For an accurate yet fast food intake recognition, this work presents a novel Machine Learning (ML) based framework that shows promising results by leveraging optimized support vector machine (SVM) classifiers. The SVM classifiers are trained on three comprehensive datasets: OREBA, FIC, and CLEMSON. The developed framework outperforms existing algorithms by achieving F1-scores of 92%, 94%, 95%, and 85% on OREBA-SHA, OREBA-DIS, FIC, and CLEMSON datasets, respectively. In order to assess the generalization aspects, the proposed SVM framework is also trained on one of the three databases while being tested on the others and achieves acceptable F1-scores in all cases. The proposed algorithm is well suited for real-time applications since inference is made using a few support vector parameters compared to thousands in peer deep neural networks models.

Plant stem tissue modeling and parameter identification using metaheuristic optimization algorithms.

Bio-impedance non-invasive measurement techniques usage is rapidly increasing in the agriculture industry. These measured impedance variations reflect tacit biochemical and biophysical changes of living and non-living tissues. Bio-impedance circuit modeling is an effective solution used in biology and medicine to fit the measured impedance. This paper proposes two new fractional-order bio-impedance plant stem models. These new models are compared with three commonly used bio-impedance fractional-order circuit models in plant modeling (Cole, Double Cole, and Fractional-order Double-shell). The two proposed models represent the characterization of the biological cellular morphology of the plant stem. Experiments are conducted on two samples of three different medical plant species from the family Lamiaceae, and each sample is measured at two inter-electrode spacing distances. Bio-impedance measurements are done using an electrochemical station (SP150) in the range of 100 Hz to 100 kHz. All employed models are compared by fitting the measured data to verify the efficiency of the proposed models in modeling the plant stem tissue. The proposed models give the best results in all inter-electrode spacing distances. Four different metaheuristic optimization algorithms are used in the fitting process to extract all models parameter and find the best optimization algorithm in the bio-impedance problems.

Parameter Estimation of Two Spiking Neuron Models With Meta-Heuristic Optimization Algorithms.

The automatic fitting of spiking neuron models to experimental data is a challenging problem. The integrate and fire model and Hodgkin-Huxley (HH) models represent the two complexity extremes of spiking neural models. Between these two extremes lies two and three differential-equation-based models. In this work, we investigate the problem of parameter estimation of two simple neuron models with a sharp reset in order to fit the spike timing of electro-physiological recordings based on two problem formulations. Five optimization algorithms are investigated; three of them have not been used to tackle this problem before. The new algorithms show improved fitting when compared with the old ones in both problems under investigation. The improvement in fitness function is between 5 and 8%, which is achieved by using the new algorithms while also being more consistent between independent trials. Furthermore, a new problem formulation is investigated that uses a lower number of search space variables when compared to the ones reported in related literature.

A flexible capacitive photoreceptor for the biomimetic retina.

Neuromorphic vision sensors have been extremely beneficial in developing energy-efficient intelligent systems for robotics and privacy-preserving security applications. There is a dire need for devices to mimic the retina's photoreceptors that encode the light illumination into a sequence of spikes to develop such sensors. Herein, we develop a hybrid perovskite-based flexible photoreceptor whose capacitance changes proportionally to the light intensity mimicking the retina's rod cells, paving the way for developing an efficient artificial retina network. The proposed device constitutes a hybrid nanocomposite of perovskites (methyl-ammonium lead bromide) and the ferroelectric terpolymer (polyvinylidene fluoride trifluoroethylene-chlorofluoroethylene). A metal-insulator-metal type capacitor with the prepared composite exhibits the unique and photosensitive capacitive behavior at various light intensities in the visible light spectrum. The proposed photoreceptor mimics the spectral sensitivity curve of human photopic vision. The hybrid nanocomposite is stable in ambient air for 129 weeks, with no observable degradation of the composite due to the encapsulation of hybrid perovskites in the hydrophobic polymer. The functionality of the proposed photoreceptor to recognize handwritten digits (MNIST) dataset using an unsupervised trained spiking neural network with 72.05% recognition accuracy is demonstrated. This demonstration proves the potential of the proposed sensor for neuromorphic vision applications.

Toward the Optimal Design and FPGA Implementation of Spiking Neural Networks.

The performance of a biologically plausible spiking neural network (SNN) largely depends on the model parameters and neural dynamics. This article proposes a parameter optimization scheme for improving the performance of a biologically plausible SNN and a parallel on-field-programmable gate array (FPGA) online learning neuromorphic platform for the digital implementation based on two numerical methods, namely, the Euler and third-order Runge-Kutta (RK3) methods. The optimization scheme explores the impact of biological time constants on information transmission in the SNN and improves the convergence rate of the SNN on digit recognition with a suitable choice of the time constants. The parallel digital implementation leads to a significant speedup over software simulation on a general-purpose CPU. The parallel implementation with the Euler method enables around 180× ( 20× ) training (inference) speedup over a Pytorch-based SNN simulation on CPU. Moreover, compared with previous work, our parallel implementation shows more than 300× ( 240× ) improvement on speed and 180× ( 250× ) reduction in energy consumption for training (inference). In addition, due to the high-order accuracy, the RK3 method is demonstrated to gain 2× training speedup over the Euler method, which makes it suitable for online training in real-time applications.

Neural Coding in Spiking Neural Networks: A Comparative Study for Robust Neuromorphic Systems.

Various hypotheses of information representation in brain, referred to as neural codes, have been proposed to explain the information transmission between neurons. Neural coding plays an essential role in enabling the brain-inspired spiking neural networks (SNNs) to perform different tasks. To search for the best coding scheme, we performed an extensive comparative study on the impact and performance of four important neural coding schemes, namely, rate coding, time-to-first spike (TTFS) coding, phase coding, and burst coding. The comparative study was carried out using a biological 2-layer SNN trained with an unsupervised spike-timing-dependent plasticity (STDP) algorithm. Various aspects of network performance were considered, including classification accuracy, processing latency, synaptic operations (SOPs), hardware implementation, network compression efficacy, input and synaptic noise resilience, and synaptic fault tolerance. The classification tasks on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets were applied in our study. For hardware implementation, area and power consumption were estimated for these coding schemes, and the network compression efficacy was analyzed using pruning and quantization techniques. Different types of input noise and noise variations in the datasets were considered and applied. Furthermore, the robustness of each coding scheme to the non-ideality-induced synaptic noise and fault in analog neuromorphic systems was studied and compared. Our results show that TTFS coding is the best choice in achieving the highest computational performance with very low hardware implementation overhead. TTFS coding requires 4x/7.5x lower processing latency and 3.5x/6.5x fewer SOPs than rate coding during the training/inference process. Phase coding is the most resilient scheme to input noise. Burst coding offers the highest network compression efficacy and the best overall robustness to hardware non-idealities for both training and inference processes. The study presented in this paper reveals the design space created by the choice of each coding scheme, allowing designers to frame each scheme in terms of its strength and weakness given a designs' constraints and considerations in neuromorphic systems.

Unsupervised Adaptive Weight Pruning for Energy-Efficient Neuromorphic Systems.

To tackle real-world challenges, deep and complex neural networks are generally used with a massive number of parameters, which require large memory size, extensive computational operations, and high energy consumption in neuromorphic hardware systems. In this work, we propose an unsupervised online adaptive weight pruning method that dynamically removes non-critical weights from a spiking neural network (SNN) to reduce network complexity and improve energy efficiency. The adaptive pruning method explores neural dynamics and firing activity of SNNs and adapts the pruning threshold over time and neurons during training. The proposed adaptation scheme allows the network to effectively identify critical weights associated with each neuron by changing the pruning threshold dynamically over time and neurons. It balances the connection strength of neurons with the previous layer with adaptive thresholds and prevents weak neurons from failure after pruning. We also evaluated improvement in the energy efficiency of SNNs with our method by computing synaptic operations (SOPs). Simulation results and detailed analyses have revealed that applying adaptation in the pruning threshold can significantly improve network performance and reduce the number of SOPs. The pruned SNN with 800 excitatory neurons can achieve a 30% reduction in SOPs during training and a 55% reduction during inference, with only 0.44% accuracy loss on MNIST dataset. Compared with a previously reported online soft pruning method, the proposed adaptive pruning method shows 3.33% higher classification accuracy and 67% more reduction in SOPs. The effectiveness of our method was confirmed on different datasets and for different network sizes. Our evaluation showed that the implementation overhead of the adaptive method regarding speed, area, and energy is negligible in the network. Therefore, this work offers a promising solution for effective network compression and building highly energy-efficient neuromorphic systems in real-time applications.

Efficient Acceleration of Stencil Applications through In-Memory Computing.

The traditional computer architectures severely suffer from the bottleneck between the processing elements and memory that is the biggest barrier in front of their scalability. Nevertheless, the amount of data that applications need to process is increasing rapidly, especially after the era of big data and artificial intelligence. This fact forces new constraints in computer architecture design towards more data-centric principles. Therefore, new paradigms such as in-memory and near-memory processors have begun to emerge to counteract the memory bottleneck by bringing memory closer to computation or integrating them. Associative processors are a promising candidate for in-memory computation, which combines the processor and memory in the same location to alleviate the memory bottleneck. One of the applications that need iterative processing of a huge amount of data is stencil codes. Considering this feature, associative processors can provide a paramount advantage for stencil codes. For demonstration, two in-memory associative processor architectures for 2D stencil codes are proposed, implemented by both emerging memristor and traditional SRAM technologies. The proposed architecture achieves a promising efficiency for a variety of stencil applications and thus proves its applicability for scientific stencil computing.

New renal haemodynamic indices can predict worsening of renal function in acute decompensated heart failure.

AIMS: Worsening of renal function (WRF) is a common complication in patients with acute decompensated heart failure (ADHF). We aimed to evaluate the role of intrarenal Doppler ultrasound (IRD) in the early prediction of WRF in this patient group. METHODS AND RESULTS: Among 90 patients (age: 57.5 ± 11.1 years; 62% male) hospitalized with ADHF, resistivity index (RI), acceleration time (AT), and pulsatility index (PI) were measured on admission and at 24 and 72 h. WRF was defined as increased serum creatinine ≥0.3 mg/dL from baseline. Adverse clinical outcomes were defined as the composite of death, use of vasopressors, and need for ultrafiltration for refractory oedema. WRF developed in 40% of patients. Mean values of renal AT, RI, and PI on admission were 59.7 ± 15, 0.717 ± 0.08, and 1.5 ± 0.48 ms, respectively. At 24 h, there was significant decrease in AT (to 56.7 ± 10 ms, P = 0.02) and renal RI (to 0.732 ± 0.07; P < 0.001); these changes were maintained up to 72 h. Renal PI showed no significant changes. Independent predictors of WRF were renal AT at 24 h and admission values of renal RI, left ventricular ejection fraction, and plasma cystatin C. Renal AT at 24 h ≥ 57.8 ms had 89% sensitivity and 70% specificity for the prediction of WRF. Independent predictors for adverse clinical outcomes were left ventricular end systolic dimension and WRF. CONCLUSIONS: Among ADHF patients receiving diuretic therapy, measurement of renal AT and RI by IRD can help identify patients at increased risk for WRF.

Biometric Identity Based on Intra-Body Communication Channel Characteristics and Machine Learning.

In this paper, we propose and validate using the Intra-body communications channel as a biometric identity. Combining experimental measurements collected from five subjects and two multi-layer tissue mimicking materials' phantoms, different machine learning algorithms were used and compared to test and validate using the channel characteristics and features as a biometric identity for subject identification. An accuracy of 98.5% was achieved, together with a precision and recall of 0.984 and 0.984, respectively, when testing the models against subject identification over results collected from the total samples. Using a simple and portable setup, this work shows the feasibility, reliability, and accuracy of the proposed biometric identity, which allows for continuous identification and verification.

An Ultra-Area-Efficient 1024-Point In-Memory FFT Processor.

Current computation architectures rely on more processor-centric design principles. On the other hand, the inevitable increase in the amount of data that applications need forces researchers to design novel processor architectures that are more data-centric. By following this principle, this study proposes an area-efficient Fast Fourier Transform (FFT) processor through in-memory computing. The proposed architecture occupies the smallest footprint of around 0.1 mm 2 inside its class together with acceptable power efficiency. According to the results, the processor exhibits the highest area efficiency ( FFT / s / area ) among the existing FFT processors in the current literature.