A Review of State-of-the-Art Mixed-Precision Neural Network Frameworks.

Mixed-precision Deep Neural Networks (DNNs) provide an efficient solution for hardware deployment, especially under resource constraints, while maintaining model accuracy. Identifying the ideal bit precision for each layer, however, remains a challenge given the vast array of models, datasets, and quantization schemes, leading to an expansive search space. Recent literature has addressed this challenge, resulting in several promising frameworks. This paper offers a comprehensive overview of the standard quantization classifications prevalent in existing studies. A detailed survey of current mixed-precision frameworks is provided, with an in-depth comparative analysis highlighting their respective merits and limitations. The paper concludes with insights into potential avenues for future research in this domain.

Efficient noise mitigation technique for quantum computing.

Quantum computers have enabled solving problems beyond the current machines' capabilities. However, this requires handling noise arising from unwanted interactions in these systems. Several protocols have been proposed to address efficient and accurate quantum noise profiling and mitigation. In this work, we propose a novel protocol that efficiently estimates the average output of a noisy quantum device to be used for quantum noise mitigation. The multi-qubit system average behavior is approximated as a special form of a Pauli Channel where Clifford gates are used to estimate the average output for circuits of different depths. The characterized Pauli channel error rates, and state preparation and measurement errors are then used to construct the outputs for different depths thereby eliminating the need for large simulations and enabling efficient mitigation. We demonstrate the efficiency of the proposed protocol on four IBM Q 5-qubit quantum devices. Our method demonstrates improved accuracy with efficient noise characterization. We report up to 88% and 69% improvement for the proposed approach compared to the unmitigated, and pure measurement error mitigation approaches, respectively.

Efficient training of spiking neural networks with temporally-truncated local backpropagation through time.

Directly training spiking neural networks (SNNs) has remained challenging due to complex neural dynamics and intrinsic non-differentiability in firing functions. The well-known backpropagation through time (BPTT) algorithm proposed to train SNNs suffers from large memory footprint and prohibits backward and update unlocking, making it impossible to exploit the potential of locally-supervised training methods. This work proposes an efficient and direct training algorithm for SNNs that integrates a locally-supervised training method with a temporally-truncated BPTT algorithm. The proposed algorithm explores both temporal and spatial locality in BPTT and contributes to significant reduction in computational cost including GPU memory utilization, main memory access and arithmetic operations. We thoroughly explore the design space concerning temporal truncation length and local training block size and benchmark their impact on classification accuracy of different networks running different types of tasks. The results reveal that temporal truncation has a negative effect on the accuracy of classifying frame-based datasets, but leads to improvement in accuracy on event-based datasets. In spite of resulting information loss, local training is capable of alleviating overfitting. The combined effect of temporal truncation and local training can lead to the slowdown of accuracy drop and even improvement in accuracy. In addition, training deep SNNs' models such as AlexNet classifying CIFAR10-DVS dataset leads to 7.26% increase in accuracy, 89.94% reduction in GPU memory, 10.79% reduction in memory access, and 99.64% reduction in MAC operations compared to the standard end-to-end BPTT. Thus, the proposed method has shown high potential to enable fast and energy-efficient on-chip training for real-time learning at the edge.

Parameter Estimation of Two Spiking Neuron Models With Meta-Heuristic Optimization Algorithms.

The automatic fitting of spiking neuron models to experimental data is a challenging problem. The integrate and fire model and Hodgkin-Huxley (HH) models represent the two complexity extremes of spiking neural models. Between these two extremes lies two and three differential-equation-based models. In this work, we investigate the problem of parameter estimation of two simple neuron models with a sharp reset in order to fit the spike timing of electro-physiological recordings based on two problem formulations. Five optimization algorithms are investigated; three of them have not been used to tackle this problem before. The new algorithms show improved fitting when compared with the old ones in both problems under investigation. The improvement in fitness function is between 5 and 8%, which is achieved by using the new algorithms while also being more consistent between independent trials. Furthermore, a new problem formulation is investigated that uses a lower number of search space variables when compared to the ones reported in related literature.

Toward the Optimal Design and FPGA Implementation of Spiking Neural Networks.

The performance of a biologically plausible spiking neural network (SNN) largely depends on the model parameters and neural dynamics. This article proposes a parameter optimization scheme for improving the performance of a biologically plausible SNN and a parallel on-field-programmable gate array (FPGA) online learning neuromorphic platform for the digital implementation based on two numerical methods, namely, the Euler and third-order Runge-Kutta (RK3) methods. The optimization scheme explores the impact of biological time constants on information transmission in the SNN and improves the convergence rate of the SNN on digit recognition with a suitable choice of the time constants. The parallel digital implementation leads to a significant speedup over software simulation on a general-purpose CPU. The parallel implementation with the Euler method enables around 180× ( 20× ) training (inference) speedup over a Pytorch-based SNN simulation on CPU. Moreover, compared with previous work, our parallel implementation shows more than 300× ( 240× ) improvement on speed and 180× ( 250× ) reduction in energy consumption for training (inference). In addition, due to the high-order accuracy, the RK3 method is demonstrated to gain 2× training speedup over the Euler method, which makes it suitable for online training in real-time applications.

A flexible capacitive photoreceptor for the biomimetic retina.

Neuromorphic vision sensors have been extremely beneficial in developing energy-efficient intelligent systems for robotics and privacy-preserving security applications. There is a dire need for devices to mimic the retina's photoreceptors that encode the light illumination into a sequence of spikes to develop such sensors. Herein, we develop a hybrid perovskite-based flexible photoreceptor whose capacitance changes proportionally to the light intensity mimicking the retina's rod cells, paving the way for developing an efficient artificial retina network. The proposed device constitutes a hybrid nanocomposite of perovskites (methyl-ammonium lead bromide) and the ferroelectric terpolymer (polyvinylidene fluoride trifluoroethylene-chlorofluoroethylene). A metal-insulator-metal type capacitor with the prepared composite exhibits the unique and photosensitive capacitive behavior at various light intensities in the visible light spectrum. The proposed photoreceptor mimics the spectral sensitivity curve of human photopic vision. The hybrid nanocomposite is stable in ambient air for 129 weeks, with no observable degradation of the composite due to the encapsulation of hybrid perovskites in the hydrophobic polymer. The functionality of the proposed photoreceptor to recognize handwritten digits (MNIST) dataset using an unsupervised trained spiking neural network with 72.05% recognition accuracy is demonstrated. This demonstration proves the potential of the proposed sensor for neuromorphic vision applications.

Neural Coding in Spiking Neural Networks: A Comparative Study for Robust Neuromorphic Systems.

Various hypotheses of information representation in brain, referred to as neural codes, have been proposed to explain the information transmission between neurons. Neural coding plays an essential role in enabling the brain-inspired spiking neural networks (SNNs) to perform different tasks. To search for the best coding scheme, we performed an extensive comparative study on the impact and performance of four important neural coding schemes, namely, rate coding, time-to-first spike (TTFS) coding, phase coding, and burst coding. The comparative study was carried out using a biological 2-layer SNN trained with an unsupervised spike-timing-dependent plasticity (STDP) algorithm. Various aspects of network performance were considered, including classification accuracy, processing latency, synaptic operations (SOPs), hardware implementation, network compression efficacy, input and synaptic noise resilience, and synaptic fault tolerance. The classification tasks on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets were applied in our study. For hardware implementation, area and power consumption were estimated for these coding schemes, and the network compression efficacy was analyzed using pruning and quantization techniques. Different types of input noise and noise variations in the datasets were considered and applied. Furthermore, the robustness of each coding scheme to the non-ideality-induced synaptic noise and fault in analog neuromorphic systems was studied and compared. Our results show that TTFS coding is the best choice in achieving the highest computational performance with very low hardware implementation overhead. TTFS coding requires 4x/7.5x lower processing latency and 3.5x/6.5x fewer SOPs than rate coding during the training/inference process. Phase coding is the most resilient scheme to input noise. Burst coding offers the highest network compression efficacy and the best overall robustness to hardware non-idealities for both training and inference processes. The study presented in this paper reveals the design space created by the choice of each coding scheme, allowing designers to frame each scheme in terms of its strength and weakness given a designs' constraints and considerations in neuromorphic systems.

Optimal charging of fractional-order circuits with Cuckoo search.

INTRODUCTION: Optimal charging of RC circuits is a well-studied problem in the integer-order domain due to its importance from economic and system temperature hazards perspectives. However, the fractional-order counterpart of this problem requires investigation. OBJECTIVES: This study aims to find approximate solutions of the most energy-efficient input charging function in fractional-order RC circuits. METHODS: This paper uses a meta-heuristic optimization technique called Cuckoo search optimizer to attain the maximum charging efficiency of three common fractional-order RC circuits. An analytical expression of the fractional capacitor voltage is suggested such that it satisfies the boundary conditions of the optimal charging problem. The problem is formulated as a fractional-order calculus of variations problem with compositional functional. The numerical solutions are obtained with the meta-heuristic optimization algorithm's help to avoid the complexities of the analytical approach. RESULTS: he efficiency surfaces and input voltage charging curves are discussed for fractional-order in the range 0.5 < α ≤ 1 . CONCLUSION: The optimized charging function can approximate the optimal charging curve using at most 4 terms. The charging time and the resistive parameters have the most dominant effect on charging efficiency at constant fractional-order α .

Unsupervised Adaptive Weight Pruning for Energy-Efficient Neuromorphic Systems.

To tackle real-world challenges, deep and complex neural networks are generally used with a massive number of parameters, which require large memory size, extensive computational operations, and high energy consumption in neuromorphic hardware systems. In this work, we propose an unsupervised online adaptive weight pruning method that dynamically removes non-critical weights from a spiking neural network (SNN) to reduce network complexity and improve energy efficiency. The adaptive pruning method explores neural dynamics and firing activity of SNNs and adapts the pruning threshold over time and neurons during training. The proposed adaptation scheme allows the network to effectively identify critical weights associated with each neuron by changing the pruning threshold dynamically over time and neurons. It balances the connection strength of neurons with the previous layer with adaptive thresholds and prevents weak neurons from failure after pruning. We also evaluated improvement in the energy efficiency of SNNs with our method by computing synaptic operations (SOPs). Simulation results and detailed analyses have revealed that applying adaptation in the pruning threshold can significantly improve network performance and reduce the number of SOPs. The pruned SNN with 800 excitatory neurons can achieve a 30% reduction in SOPs during training and a 55% reduction during inference, with only 0.44% accuracy loss on MNIST dataset. Compared with a previously reported online soft pruning method, the proposed adaptive pruning method shows 3.33% higher classification accuracy and 67% more reduction in SOPs. The effectiveness of our method was confirmed on different datasets and for different network sizes. Our evaluation showed that the implementation overhead of the adaptive method regarding speed, area, and energy is negligible in the network. Therefore, this work offers a promising solution for effective network compression and building highly energy-efficient neuromorphic systems in real-time applications.

Realization of fractional-order capacitor based on passive symmetric network.

In this paper, a new realization of the fractional capacitor (FC) using passive symmetric networks is proposed. A general analysis of the symmetric network that is independent of the internal impedance composition is introduced. Three different internal impedances are utilized in the network to realize the required response of the FC. These three cases are based on either a series RC circuit, integer Cole-impedance circuit, or both. The network size and the values of the passive elements are optimized using the minimax and least m th optimization techniques. The proposed realizations are compared with well-known realizations achieving a reasonable performance with a phase error of approximately 2 o . Since the target of this emulator circuit is the use of off-the-shelf components, Monte Carlo simulations with 5 % tolerance in the utilized elements are presented. In addition, experimental measurements of the proposed capacitors are preformed, therein showing comparable results with the simulations. The proposed realizations can be used to emulate the FC for experimental verifications of new fractional-order circuits and systems. The functionality of the proposed realizations is verified using two oscillator examples: a fractional-order Wien oscillator and a relaxation oscillator.