Memristor-CMOS Hybrid Circuits Implementing Event-Driven Neural Networks for Dynamic Vision Sensor Camera.

For processing streaming events from a Dynamic Vision Sensor camera, two types of neural networks can be considered. One are spiking neural networks, where simple spike-based computation is suitable for low-power consumption, but the discontinuity in spikes can make the training complicated in terms of hardware. The other one are digital Complementary Metal Oxide Semiconductor (CMOS)-based neural networks that can be trained directly using the normal backpropagation algorithm. However, the hardware and energy overhead can be significantly large, because all streaming events must be accumulated and converted into histogram data, which requires a large amount of memory such as SRAM. In this paper, to combine the spike-based operation with the normal backpropagation algorithm, memristor-CMOS hybrid circuits are proposed for implementing event-driven neural networks in hardware. The proposed hybrid circuits are composed of input neurons, synaptic crossbars, hidden/output neurons, and a neural network's controller. Firstly, the input neurons perform preprocessing for the DVS camera's events. The events are converted to histogram data using very simple memristor-based latches in the input neurons. After preprocessing the events, the converted histogram data are delivered to an ANN implemented using synaptic memristor crossbars. The memristor crossbars can perform low-power Multiply-Accumulate (MAC) calculations according to the memristor's current-voltage relationship. The hidden and output neurons can convert the crossbar's column currents to the output voltages according to the Rectified Linear Unit (ReLU) activation function. The neural network's controller adjusts the MAC calculation frequency according to the workload of the event computation. Moreover, the controller can disable the MAC calculation clock automatically to minimize unnecessary power consumption. The proposed hybrid circuits have been verified by circuit simulation for several event-based datasets such as POKER-DVS and MNIST-DVS. The circuit simulation results indicate that the neural network's performance proposed in this paper is degraded by as low as 0.5% while saving as much as 79% in power consumption for POKER-DVS. The recognition rate of the proposed scheme is lower by 0.75% compared to the conventional one, for the MNIST-DVS dataset. In spite of this little loss, the power consumption can be reduced by as much as 75% for the proposed scheme.

Assessing the Role of Yarn Placement in Plated Knit Strain Sensors: A Detailed Study of Their Electromechanical Properties and Applicability in Bending Cycle Monitoring.

In this study, we explore how the strategic positioning of conductive yarns influences the performance of plated knit strain sensors fabricated using commercial knitting machines with both conductive and non-conductive yarns. Our study reveals that sensors with conductive yarns located at the rear, referred to as 'purl plated sensors', exhibit superior performance in comparison to those with conductive yarns at the front, or 'knit plated sensors'. Specifically, purl plated sensors demonstrate a higher sensitivity, evidenced by a gauge factor ranging from 3 to 18, and a minimized strain delay, indicated by a 1% strain in their electromechanical response. To elucidate the mechanisms behind these observations, we developed an equivalent circuit model. This model examines the role of contact resistance within varying yarn configurations on the sensors' sensitivity, highlighting the critical influence of contact resistance in conductive yarns subjected to wale-wise stretching on sensor responsiveness. Furthermore, our findings illustrate that the purl plated sensors benefit from the vertical movement of non-conductive yarns, which promotes enhanced contact between adjacent conductive yarns, thereby improving both the stability and sensitivity of the sensors. The practicality of these sensors is confirmed through bending cycle tests with an in situ monitoring system, showcasing the purl plated sensors' exceptional reproducibility, with a standard deviation of 0.015 across 1000 cycles, and their superior sensitivity, making them ideal for wearable devices designed for real-time joint movement monitoring. This research highlights the critical importance of conductive yarn placement in sensor efficacy, providing valuable guidance for crafting advanced textile-based strain sensors.

Memristor Crossbar Circuits Implementing Equilibrium Propagation for On-Device Learning.

Equilibrium propagation (EP) has been proposed recently as a new neural network training algorithm based on a local learning concept, where only local information is used to calculate the weight update of the neural network. Despite the advantages of local learning, numerical iteration for solving the EP dynamic equations makes the EP algorithm less practical for realizing edge intelligence hardware. Some analog circuits have been suggested to solve the EP dynamic equations physically, not numerically, using the original EP algorithm. However, there are still a few problems in terms of circuit implementation: for example, the need for storing the free-phase solution and the lack of essential peripheral circuits for calculating and updating synaptic weights. Therefore, in this paper, a new analog circuit technique is proposed to realize the EP algorithm in practical and implementable hardware. This work has two major contributions in achieving this objective. First, the free-phase and nudge-phase solutions are calculated by the proposed analog circuits simultaneously, not at different times. With this process, analog voltage memories or digital memories with converting circuits between digital and analog domains for storing the free-phase solution temporarily can be eliminated in the proposed EP circuit. Second, a simple EP learning rule relying on a fixed amount of conductance change per programming pulse is newly proposed and implemented in peripheral circuits. The modified EP learning rule can make the weight update circuit practical and implementable without requiring the use of a complicated program verification scheme. The proposed memristor conductance update circuit is simulated and verified for training synaptic weights on memristor crossbars. The simulation results showed that the proposed EP circuit could be used for realizing on-device learning in edge intelligence hardware.

Area-Efficient Mapping of Convolutional Neural Networks to Memristor Crossbars Using Sub-Image Partitioning.

Memristor crossbars can be very useful for realizing edge-intelligence hardware, because the neural networks implemented by memristor crossbars can save significantly more computing energy and layout area than the conventional CMOS (complementary metal-oxide-semiconductor) digital circuits. One of the important operations used in neural networks is convolution. For performing the convolution by memristor crossbars, the full image should be partitioned into several sub-images. By doing so, each sub-image convolution can be mapped to small-size unit crossbars, of which the size should be defined as 128 × 128 or 256 × 256 to avoid the line resistance problem caused from large-size crossbars. In this paper, various convolution schemes with 3D, 2D, and 1D kernels are analyzed and compared in terms of neural network's performance and overlapping overhead. The neural network's simulation indicates that the 2D + 1D kernels can perform the sub-image convolution using a much smaller number of unit crossbars with less rate loss than the 3D kernels. When the CIFAR-10 dataset is tested, the mapping of sub-image convolution of 2D + 1D kernels to crossbars shows that the number of unit crossbars can be reduced almost by 90% and 95%, respectively, for 128 × 128 and 256 × 256 crossbars, compared with the 3D kernels. On the contrary, the rate loss of 2D + 1D kernels can be less than 2%. To improve the neural network's performance more, the 2D + 1D kernels can be combined with 3D kernels in one neural network. When the normalized ratio of 2D + 1D layers is around 0.5, the neural network's performance indicates very little rate loss compared to when the normalized ratio of 2D + 1D layers is zero. However, the number of unit crossbars for the normalized ratio = 0.5 can be reduced by half compared with that for the normalized ratio = 0.

Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars.

To overcome the limitations of CMOS digital systems, emerging computing circuits such as memristor crossbars have been investigated as potential candidates for significantly increasing the speed and energy efficiency of next-generation computing systems, which are required for implementing future AI hardware. Unfortunately, manufacturing yield still remains a serious challenge in adopting memristor-based computing systems due to the limitations of immature fabrication technology. To compensate for malfunction of neural networks caused from the fabrication-related defects, a new crossbar training scheme combining the synapse-aware with the neuron-aware together is proposed in this paper, for optimizing the defect map size and the neural network's performance simultaneously. In the proposed scheme, the memristor crossbar's columns are divided into 3 groups, which are the severely-defective, moderately-defective, and normal columns, respectively. Here, each group is trained according to the trade-off relationship between the neural network's performance and the hardware overhead of defect-tolerant training. As a result of this group-based training method combining the neuron-aware with the synapse-aware, in this paper, the new scheme can be successful in improving the network's performance better than both the synapse-aware and the neuron-aware while minimizing its hardware burden. For example, when testing the defect percentage = 10% with MNIST dataset, the proposed scheme outperforms the synapse-aware and the neuron-aware by 3.8% and 3.4% for the number of crossbar's columns trained for synapse defects = 10 and 138 among 310, respectively, while maintaining the smaller memory size than the synapse-aware. When the trained columns = 138, the normalized memory size of the synapse-neuron-aware scheme can be smaller by 3.1% than the synapse-aware.

Memristor-CMOS Hybrid Neuron Circuit with Nonideal-Effect Correction Related to Parasitic Resistance for Binary-Memristor-Crossbar Neural Networks.

Voltages and currents in a memristor crossbar can be significantly affected due to nonideal effects such as parasitic source, line, and neuron resistance. These nonideal effects related to the parasitic resistance can cause the degradation of the neural network's performance realized with the nonideal memristor crossbar. To avoid performance degradation due to the parasitic-resistance-related nonideal effects, adaptive training methods were proposed previously. However, the complicated training algorithm could add a heavy computational burden to the neural network hardware. Especially, the hardware and algorithmic burden can be more serious for edge intelligence applications such as Internet of Things (IoT) sensors. In this paper, a memristor-CMOS hybrid neuron circuit is proposed for compensating the parasitic-resistance-related nonideal effects during not the training phase but the inference one, where the complicated adaptive training is not needed. Moreover, unlike the previous linear correction method performed by the external hardware, the proposed correction circuit can be included in the memristor crossbar to minimize the power and hardware overheads for compensating the nonideal effects. The proposed correction circuit has been verified to be able to restore the degradation of source and output voltages in the nonideal crossbar. For the source voltage, the average percentage error of the uncompensated crossbar is as large as 36.7%. If the correction circuit is used, the percentage error in the source voltage can be reduced from 36.7% to 7.5%. For the output voltage, the average percentage error of the uncompensated crossbar is as large as 65.2%. The correction circuit can improve the percentage error in the output voltage from 65.2% to 8.6%. Almost the percentage error can be reduced to ~1/7 if the correction circuit is used. The nonideal memristor crossbar with the correction circuit has been tested for MNIST and CIFAR-10 datasets in this paper. For MNIST, the uncompensated and compensated crossbars indicate the recognition rate of 90.4% and 95.1%, respectively, compared to 95.5% of the ideal crossbar. For CIFAR-10, the nonideal crossbars without and with the nonideal-effect correction show the rate of 85.3% and 88.1%, respectively, compared to the ideal crossbar achieving the rate as large as 88.9%.

Exploiting defective RRAM array as synapses of HTM spatial pooler with boost-factor adjustment scheme for defect-tolerant neuromorphic systems.

A crossbar array architecture employing resistive switching memory (RRAM) as a synaptic element accelerates vector-matrix multiplication in a parallel fashion, enabling energy-efficient pattern recognition. To implement the function of the synapse in the RRAM, multilevel resistance states are required. More importantly, a large on/off ratio of the RRAM should be preferentially obtained to ensure a reasonable margin between each state taking into account the inevitable variability caused by the inherent switching mechanism. The on/off ratio is basically adjusted in two ways by modulating measurement conditions such as compliance current or voltage pulses modulation. The latter technique is not only more suitable for practical systems, but also can achieve multiple states in low current range. However, at the expense of applying a high negative voltage aimed at enlarging the on/off ratio, a breakdown of the RRAM occurs unexpectedly. This stuck-at-short fault of the RRAM adversely affects the recognition process based on reading and judging each column current changed by the multiplication of the input voltage and resistance of the RRAM in the array, degrading the accuracy. To address this challenge, we introduce a boost-factor adjustment technique as a fault-tolerant scheme based on simple circuitry that eliminates the additional process to identify specific locations of the failed RRAMs in the array. Spectre circuit simulation is performed to verify the effect of the scheme on Modified National Institute of Standards and Technology dataset using convolutional neural networks in non-ideal crossbar arrays, where experimentally observed imperfective RRAMs are configured. Our results show that the recognition accuracy can be maintained similar to the ideal case because the interruption of the failure is suppressed by the scheme.

Hybrid Circuit of Memristor and Complementary Metal-Oxide-Semiconductor for Defect-Tolerant Spatial Pooling with Boost-Factor Adjustment.

Hierarchical Temporal Memory (HTM) has been known as a software framework to model the brain's neocortical operation. However, mimicking the brain's neocortical operation by not software but hardware is more desirable, because the hardware can not only describe the neocortical operation, but can also employ the brain's architectural advantages. To develop a hybrid circuit of memristor and Complementary Metal-Oxide-Semiconductor (CMOS) for realizing HTM's spatial pooler (SP) by hardware, memristor defects such as stuck-at-faults and variations should be considered. For solving the defect problem, we first show that the boost-factor adjustment can make HTM's SP defect-tolerant, because the false activation of defective columns are suppressed. Second, we propose a memristor-CMOS hybrid circuit with the boost-factor adjustment to realize this defect-tolerant SP by hardware. The proposed circuit does not rely on the conventional defect-aware mapping scheme, which cannot avoid the false activation of defective columns. For the Modified subset of National Institute of Standards and Technology (MNIST) vectors, the boost-factor adjusted crossbar with defects = 10% shows a rate loss of only ~0.6%, compared to the ideal crossbar with defects = 0%. On the contrary, the defect-aware mapping without the boost-factor adjustment demonstrates a significant rate loss of ~21.0%. The energy overhead of the boost-factor adjustment is only ~0.05% of the programming energy of memristor synapse crossbar.

Partial-Gated Memristor Crossbar for Fast and Power-Efficient Defect-Tolerant Training.

A real memristor crossbar has defects, which should be considered during the retraining time after the pre-training of the crossbar. For retraining the crossbar with defects, memristors should be updated with the weights that are calculated by the back-propagation algorithm. Unfortunately, programming the memristors takes a very long time and consumes a large amount of power, because of the incremental behavior of memristor's program-verify scheme for the fine-tuning of memristor's conductance. To reduce the programming time and power, the partial gating scheme is proposed here to realize the partial training, where only some part of neurons are trained, which are more responsible in the recognition error. By retraining the part, rather than the entire crossbar, the programming time and power of memristor crossbar can be significantly reduced. The proposed scheme has been verified by CADENCE circuit simulation with the real memristor's Verilog-A model. When compared to retraining the entire crossbar, the loss of recognition rate of the partial gating scheme has been estimated only as small as 2.5% and 2.9%, for the MNIST and CIFAR-10 datasets, respectively. However, the programming time and power can be saved by 86% and 89.5% than the 100% retraining, respectively.

Memristor-CMOS Hybrid Circuit for Temporal-Pooling of Sensory and Hippocampal Responses of Cortical Neurons.

As a software framework, Hierarchical Temporal Memory (HTM) has been developed to perform the brain's neocortical functions, such as spatial and temporal pooling. However, it should be realized with hardware not software not only to mimic the neocortical function but also to exploit its architectural benefit. To do so, we propose a new memristor-CMOS (Complementary Metal-Oxide-Semiconductor) hybrid circuit of temporal-pooling here, which is composed of the input-layer and output-layer neurons mimicking the neocortex. In the hybrid circuit, the input-layer neurons have the proximal and basal/distal dendrites to combine sensory information with the temporal/location information from the brain's hippocampus. Using the same crossbar architecture, the output-layer neurons can perform a prediction by integrating the temporal information on the basal/distal dendrites. For training the proposed circuit, we used only simple Hebbian learning, not the complicated backpropagation algorithm. Due to the simple hardware of Hebbian learning, the proposed hybrid circuit can be very suitable to online learning. The proposed memristor-CMOS hybrid circuit has been verified by the circuit simulation using the real memristor model. The proposed circuit has been verified to predict both the ordinal and out-of-order sequences. In addition, the proposed circuit has been tested with the external noise and memristance variation.

Asymmetrical Training Scheme of Binary-Memristor-Crossbar-Based Neural Networks for Energy-Efficient Edge-Computing Nanoscale Systems.

For realizing neural networks with binary memristor crossbars, memristors should be programmed by high-resistance state (HRS) and low-resistance state (LRS), according to the training algorithms like backpropagation. Unfortunately, it takes a very long time and consumes a large amount of power in training the memristor crossbar, because the program-verify scheme of memristor-programming is based on the incremental programming pulses, where many programming and verifying pulses are repeated until the target conductance. Thus, this reduces the programming time and power is very essential for energy-efficient and fast training of memristor networks. In this paper, we compared four different programming schemes, which are F-F, C-F, F-C, and C-C, respectively. C-C means both HRS and LRS are coarse-programmed. C-F has the coarse-programmed HRS and fine LRS, respectively. F-C is vice versa of C-F. In F-F, both HRS and LRS are fine-programmed. Comparing the error-energy products among the four schemes, C-F shows the minimum error with the minimum energy consumption. The asymmetrical coarse HRS and fine LRS can reduce the time and energy during the crossbar training significantly, because only LRS is fine-programmed. Moreover, the asymmetrical C-F can maintain the network's error as small as F-F, which is due to the coarse-programmed HRS that slightly degrades the error.

Time-Shared Twin Memristor Crossbar Reducing the Number of Arrays by Half for Pattern Recognition.

In this paper, we propose a new time-shared twin memristor crossbar for pattern-recognition applications. By sharing two memristor arrays at different time, the number of memristor arrays can be reduced by half, saving the crossbar area by half, too. To implement the time-shared twin memristor crossbar, we also propose CMOS time-shared subtractor circuit, in this paper. The operation of the time-shared twin memristor crossbar is verified using 3 × 3 memristor array which is made of aluminum film and carbon fiber. Here, the crossbar array is programmed to store three different patterns. When we apply three different input vectors to the array, we can verify that the input vectors are well recognized by the proposed crossbar. Moreover, the proposed crossbar is tested for the recognition of complicated gray-scale images. Here, 10 images with 32 × 32 pixels are applied to the proposed crossbar. The simulation result verifies that the input images are recognized well by the proposed crossbar, even though the noise level of each image is varied from -10 to +10 dB.

Comparative Study on Statistical-Variation Tolerance Between Complementary Crossbar and Twin Crossbar of Binary Nano-scale Memristors for Pattern Recognition.

This paper performs a comparative study on the statistical-variation tolerance between two crossbar architectures which are the complementary and twin architectures. In this comparative study, 10 greyscale images and 26 black-and-white alphabet characters are tested using the circuit simulator to compare the recognition rate with varying statistical variation and correlation parameters.As with the simulation results of 10 greyscale image recognitions, the twin crossbar shows better recognition rate by 4 % on average than the complementary one, when the inter-array correlation = 1 and intra-array correlation = 0. When the inter-array correlation = 1 and intra-array correlation = 1, the twin architecture can recognize better by 5.6 % on average than the complementary one.Similarly, when the inter-array correlation = 1 and intra-array correlation = 0, the twin architecture can recognize 26 alphabet characters better by 4.5 % on average than the complementary one. When the inter-array correlation = 1 and intra-array correlation = 1, the twin architecture is better by 6 % on average than the complementary one. By summary, we can conclude that the twin crossbar is more robust than the complementary one under the same amounts of statistical variation and correlation.

Neuromorphic crossbar circuit with nanoscale filamentary-switching binary memristors for speech recognition.

In this paper, a neuromorphic crossbar circuit with binary memristors is proposed for speech recognition. The binary memristors which are based on filamentary-switching mechanism can be found more popularly and are easy to be fabricated than analog memristors that are rare in materials and need a more complicated fabrication process. Thus, we develop a neuromorphic crossbar circuit using filamentary-switching binary memristors not using interface-switching analog memristors. The proposed binary memristor crossbar can recognize five vowels with 4-bit 64 input channels. The proposed crossbar is tested by 2,500 speech samples and verified to be able to recognize 89.2% of the tested samples. From the statistical simulation, the recognition rate of the binary memristor crossbar is estimated to be degraded very little from 89.2% to 80%, though the percentage variation in memristance is increased very much from 0% to 15%. In contrast, the analog memristor crossbar loses its recognition rate significantly from 96% to 9% for the same percentage variation in memristance.

Device/Circuit co-design guide for passive memristor array with non-linear current-voltage behavior.

In this paper, the non-linearity in memristor's current-voltage relationship that can affect half-selected cells is analyzed. From the simulation, for the V(DD)/2 scheme, if the non-linear coefficient is larger than 8, unwanted resistance loss during the write time can be suppressed less than 10%. Comparing the V(DD)/2 and V(DD)/3 scheme, the V(DD)/2 scheme can reduce the current consumption by 2 orders of magnitude with little larger resistance change in half-selected cells than the V(DD)/3.

Small-area and compact CMOS emulator circuit for CMOS/nanoscale memristor co-design.

In this paper, a CMOS emulator circuit that can reproduce nanoscale memristive behavior is proposed. The proposed emulator circuit can mimic the pinched hysteresis loops of nanoscale memristor memory's current-voltage relationship without using any resistor array, complicated circuit blocks, etc. that may occupy very large layout area. Instead of using a resistor array, other complicated circuit blocks, etc., the proposed emulator circuit can describe the nanoscale memristor's current-voltage relationship using a simple voltage-controlled resistor, where its resistance can be programmed by the stored voltage at the state variable capacitor. Comparing the layout area between the previous emulator circuit and the proposed one, the layout area of the proposed emulator circuit is estimated to be 32 times smaller than the previous emulator circuit. The proposed CMOS emulator circuit of nanoscale memristor memory will be very useful in developing hybrid circuits of CMOS/nanoscale memristor memory.

SPICE macromodel and CMOS emulator for memristors.

In this paper, a new SPICE macromodel and CMOS emulator for memristors are proposed and verified to fit to the memristor's model equation very well in the entire range of memristor's resistance from the RESET state to the SET state. Compared with the memristor's model equation, average percentage errors in the new SPICE macromodel and in the 4-bit CMOS emulator are less than 0.5% and 0.9%, respectively. In addition, the CMOS emulator for memristors which can be implemented by a CMOS circuit will be very useful to design and verify various peripheral circuits for memristor applications particularly when the memristor fabrication process is not ready.